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ABSTRACT 

An empirical assesssaent of the space distortion 
properties of two prototypic hierarchical clustering procedures is 
given in tens of an occupancy model developed from combinatorics* 
Osing one simple example, the single«*link and complete^link 
clustering strategies now in common use in the behavioral sciences 
are empirically shown to be space contracting and space dilating, 
respectively* An extension to an intermediate r**diameter clustering 
criterion defined in between the complete**lin)c and single«link 
extremes is discussed briefly along with a technique for 
preprocessing the original proximity measures to make the results 
obtained from these two extreme clustering procedures more consistent 
with the data. (Author) 
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HIERARCHICAL CLUSTERING AND THE CONCEPT OF SPACE DISTORTION 

Abstract 

An enpirical assessment o£ the space distortion properties of two 
prototypic hierarchical clustering procedures is given in terns o£ an 
occi;5>ancy model developed from combinatorics. Using one simple exmple^ 
the single-link and complete-link clustering strategies now in common 
use in the behavioral sciences are eiipirically shown to be space contract- 
ing and space dilating, respectively. An extension to an intermediate 
r-diameter clustering criterion defined in between the canplete-link and 
single-link extremes is discussed briefly along with a tedinique for pre- 
processing the original proximity measures to make the results obtained 
from these two extreme clustering procedures more consistent with the 
data. 
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HIERARailCAL CLUSl^ERING AND IHE CONCEPT OF SPACE DISTORTION 

1. INTRODUCTION 

One of the basic properties that supposedly characterizes all hier- 
archical clustering methods can be defined in terms of a particialar 
strategy's ability to conserve "space" in the region surrounding a new 
subset fomed during the clustering process. Although the necessary 
concepts introduced by Lance and Williams (1967) are obviously vague and 
lack fomfial or precise definitions of any sort, the lasic idea of a space 
conserving or space distorting clustering procedure does seem to have sub- 
stantial intuitive appeal, especially for an applied researcher well- 
versed in the various techniques for partitioning a set of objects into 
homogeneous subgroups. The intent of this paper is to concentrate on tw 
sc- called space distorting clustering strategies now in general use in 
the behavioral sciences, commonly called the single-link and the complete- 
liri^ methods. The discussion below offers one possible combinatorial 
framework for characterizing a space distorting strategy, and in particu- 
lar, for the terms "space contracting" and "space dilating" used in connec- 
tion with the prototypic single- link and complete- linlc clustering schemes, 
respectively. 
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2. CLUSTERING PROCEDURES 

To give a brief background summary on clustering, suppose s(«,0 
J a positive real -valued symmetric function on S x S, where S is a 
set of n objects {o^^,. . . ,0^^}; intuitively, the proximity function s(-,0 
assigns smaller numerical values to the more similar pairs of objects, 
where it is assumed chat s(o^,Oj) = 0 if o^ = o^. Both the sing^le-link 
and the complete-link clustering procedures construct a sequence of 
partitions of S, (^o.---.Vl^' ^ ^'^^^ trivial partition 

containing each object in a separate class, includes all objects in 
a single class, and Jlj^^.-|^ is obtained from by uniting two of the sub- 
sets in Jlj^. In particular, the complete-link hierarchy is defined by 
uniting those two object classes in with the minimum resulting dia- 
meter, where the tern "diameter" is defir.ed as the maximum of s(-,0 over 
jjll object pairs within a subset. The single-link method, on the other 
hand, employs an exact opposite criterion and unites those two subsets 
in that minimize s(' ,•) over all object pairs in which each member of 
the pair belongs to only one of the t^>ro subsets to be joined. 

A number of enpirical evaluations of the single-link criterion have 
documented a chaining characteristic of the method, in the sense that 
new partitions in the sequence tend to be formed by uniting single ele- 
ments with pre-existing subsets (for exmple, see Wishart, 1969). In 
other words , at a particular partition level the subsets are generally 
large and as a new subset is foimed, it appears to move nearer to some 
or to all of the remaining members of S, and consequently, the region 
around the subset appears to contract. For the complete-link procedure, 
however, the subsets within a partition tend to be more equal in size. 



and as new subsets are fonned, they appear to move further away from the 
remaining subsets in the partition. Thus, the region around a newly con- 
structed subset is said to dilate and new members of a partition are 
feimed by uniting the smaller subsets within the previoxis level. 

The concepts of "region," "space," and so forth used in the above 
discussion may be intuitively meaningful but fail to convey very precise 
pioperties. Consequently, as an alternative approach given below, the 
anphasis will be on operational izing the space-distortion ideas through 
the number and type of subsets formed r.t a specific partition level by 
the two clustering techniques. In pati:icular, a space contracting strat- 
egy- should produce, on the average, a large subset and many small ones at 
eacli partition level, and attempt to minimize the number of subsets con- 
taining more than a single object. Conversely, a space dilating technique 
should construct subsets of about the same size and maximize the number 
of subsets containing more than one object at each partition level. 



4 



3. ASSESSING SPACE DISTORTION 

As mentioned above, one o£ the ways of evaluating the space conserv- 
ing or space distorting properties of a clustering method is in teims of 
the type of subsets generally produced at a specific partition level in 
the hierarchy. Such an evaluation, however, requires some fixed set of 
functions s(*,0 that vdll be used for developing a frame of reference to 
make the phrase "generally produce'* more precise. For example, since 
both the single-link and the complete-link procedures depend solely upon 
the rank order of the object pairs specified by the proximity measures, 
the set of functions s(*,0 that vdll be employed here corresponds to the 
set of all assignments of the distinct integers from 1 to n(n-l)/2 to a 
fixed ordering of the n(n-l)/2 object pairs. Thus, for all [n(n-l)/2]! 
possible assignments of rank, the complete-link and the single-link strat- 
egies may be used to obtain the distributions of subset types produced at 
the various partition levels. To carry the discussion one step further, 
these distributions can then be compared to an expected distribution that 
would be obtained if the clustering procedure were space conserving; in 
fact, the differences between the obtained distributions and this standard 
could be incorporated into some fomal index of the space distortion pro- 
pensity of the clustering method. 

What is still lacking in this presentation is any specification of 
vtfiat the standard distribution should be at a particular partition level. 
Within the rank order framework, however, there is an obvious suggestion 
that can be developed from the enormous literature on occupancy problons. 
To be more precise, suppose there are n objects in S aiid the partition 
level under consideration consists of m subsets, i.e., the partition I 
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at level n-m in the sequence i^^f • • • f^^^i) • Th^ number of different ways 



in vtfiich the n distinguishable objects can be distributed into the m indis 
tinguishable cells of the partition is S(n,m), a Stirling number of the 
second kind (Riordan, 1958). Consequently, if apply the occupancy 
model as a standard for a space -conserving procedure and use our set of 
all possible [n(n-l)/2]! assignments of integer ranks to the object pairs, 
a space conserving clustering strategy should produce partitions at level 
n-m of the fom {n-,...,n } according to the distribution 



where n^ >^ 1 is the number of objects in the i subset, 1 £ i £m; 

Ln- = n, and b = n t!. In this context, T is a set of integers con- 
tei 

taining the number of subsets of size n^, 1 £ i <^m, but vfliich does not 
count a set of tied subsets more than once. As an example that will be 
carried out in more detail below, suppose n = 12 and m = 3. Then, S(12,3) » 
86,526 and the probability of obtaining the partition {6,3,3} is 



To make the computational burden somev^t more manageable, the fol- 
lowing discussion is limited to the case of n « 12 and m = 3. Further- 
more, instead of stuclying all [12*11/2]! possible permutations of the 
integer ranks for the object pairs, a sample of 1000 peimutations was 
selected randomly and with replacement, and for each such assignment the 
complete-link and single-link partitions obtained at level 9, i.e.. 




(2! 86,526) 



= .107. 
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when m = 3. Table 1 presents the theoretical distribution over the parti- 
tion types calculated from the general expression given above, and Table 
2 presents the empirical distributions obtained from the two clustering 
procedures xasing the 1000 randanly chosen permutations of the integer 
ranks. Table 3 compares the enpirical and the theoretical distributions 
obtained by collapsing over partitions in which the largest partition 
class has the same number of objects • 

Tables 1, 2, and 3 h- 

Compared to the reference distribution generated under the occupancy 

model. Tables 2 and 3 clearly indicate the space distortion propensities 

of the two clustering strategies under study. As an illustration, if n 

and m are fixed, the random variable X , defined as the size of the 

largest partition class, has an expected value of 5.70 under the occupancy 

model. Since the sample means corresponding to the 1000 observations on 

X for the complete-link scheme is 5.55 and 9.06 for the complete-link, 
n,m 

the single-link procedure is obviously biased towards the construction of 
larger partition classes whereas the complete-link criterion tends to over- 
equalize the distribution of objects into the m subsets of the decomposi- 
tion. The bias in the single-link method is very lai:ge, and rather sur- 
prisingly, the bias in the complete -link method is slight although still 
evident. 
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4. r-DIAMETER HIERARCHICAL CLUSTERING 

Although many suggestions have been made for relaxing the partition- 
ing criterion used in the complete-link method, or conversely, strength- 
ening the single-link criterion, only a very few of these proposals main- 
tain a sole dependence upon the rank order of the proximity values. Two 
of the most promising of these new approaches that, in addition, can still 
construct disjoint partition classes at each level of the hierarchy are 
due to Ling (1975) and Matula (1972). Ling uses the graph-theoretic con- 
cept of minimum degree and Matula emphasizes k-line connectivity and graph 
cohesiveness. The interested reader is referred to Hubert (1974b) for an 
extensive review of the literature relating to these techniques as well 
as for a more general background for the method suggested below. 

As an alternative characterization of complete-link clustering, sup- 
pose the partition has been obtained and the partition is to be 
foimed by uniting the two subsets in with the minimum resultant dia- 
metei. As mentioned above, the diameter of a subset of S is defined as 
the maximum proximity measure over all object pairs within the subset. 
As a convenience in defining the generalizations below, this particular 
concept of a diameter will be referred to as 1-diameter (see Hubert, 1974b). 
The concept of an r-diameter subset follows naturally from Luce's (1950) 
notion of a generalized clique and Peay*s (1970) application of .this idea 
to clustering by means of hierarchical overlapping subsets. In particular, 
the r-diameter of a subset of S is defined as the minimum proximity 
measure Q over all object pairs in that satisfies the following pro- 
perty: 
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for any two objects o. and o- eS' , there is a sequence of not 

^0 ^t 

necessarily distinct objects o. ,o. ,...,o. ,o. such that 

^0 ^1 ^r-1 h 
0. eS', 0 < j < t, and s(o. ,o. ) < Q, 0 < j ' < t-1. 

Clearly, r-diameter hierarchical clustering can be defined by unit- 
ing those two subsets in Siy, to form i^^^^ that have a minimum resulting 
r-diameter. IVhat is particularly interesting about r-diameter clustering, 
however, is the natural relation this generalization has to the single- 
link and the complete-link methods; specifically, r-diameter hierarchical 
clustering depends only upon the rank order of the proximity measures, pro- 
duces a partition at each level of the hierarchy, and corresponds to 
complete-link clustering for r = 1 and to single- link clustering for 
r - n - 1. 

From a computational point of view, the ability to perfoim any r- 
diameter clustering requires only a complete-link algorithm and a set of 
modified initial proximity values. More precisely, an r-diameter parti- 
tion hierarchy is a complete-link partition hierarchy using a transformed 
proximity function 

s^^^(o.,o.), 

fkl fk) 
defined as ":he (i,j) element in a matrix ^; the matrix ^ is con- 

structed recursively in the standard manner, i.e., ^ = ^ @ M, 

where the operation © is specified as follows: 

suppose = {a. .}, B^^ = {b..}, then 
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A similar matrix relationship is used by Peay (1970) to define a general- 
ization of a hierarchical clique detection program using Luce's (1950) 
concept of a generalized clique. 

Returning to the preAaous discussion of space-distorting clustering 
methods, the notion of an r-diameter subset provides a tentative compromise. 
The complete- link (1-diameter) and single- link (n-1 diameter) criteria 
can be viewed as foiming the two extremes of a space distortion "continuum**; 
an/ r-diameter clustering procedure lies at some point along this continuum, 
and as r increases, the strategies become less space dilating and more space 
contracting. Consequently, it is up to the applied researcher to choose 
that particular point along the continuum that satisfies his specific 
data analytic requirements; this vagueness is necessaiy since no value of 
r will provide the general space conserving procedure for all partition 
levels and for all values of n. 

As an example of the decision a user faces in applying the r-diameter 
concept, some famous published data from Rao (1952) on thi similarity of 
12 Indian Castes was clustered using all values of r. Hierarchies differ- 
e ; from those obtained with the single-link and complete-link methods 
were constructed for values of r from 2 to 4; the single-link hierarchy 
was consistently produced for r >_ 5, and the complete-link hierarchy for 
r < 1. Table 4 presents the various partitions obtained at level 7, 
which corresponds to the level of Rao's "optimal" partition also obtained 
with tiie 1-diameter (complete -link) strategy. The progression as r in- 
creases from fairly uniform sized subsets to a single elementary subset is 
obvious . 

Table 4 here 
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In general, any application of hierarchical clustering will produce 
some value of r, say Tq, such that for any value of r greater than or 
equal to r^, r-diameter hierarchical clustering will construct the single- 
link hierarchy. Similarly, there will be some value of r, say r^, such 
such that when r is less than or equal to r^^, the complete -link hierarchy 
is obtained. The size of the difference r^-rj^ can m-.^dered a "mea- 
sure" of how divergent the results obtained from the complete-link and 
single- link hierarchical clustering methods will be when used on the same 
set of data. In fact, using the paradigm discussed by Hubert (1974a), 
the san^le distribution of r^-r-^ could be approximated for each n under a 
hypothesis of randomness in the assignment of proximity ranks the ob- 
ject pairs. Although these distributioris could be of value is assessing 
the "randomness" of the basic set of proximity measures, they would be 
someis'hat expensive to obtain and the topic is not persued here. 
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5. THE "SMOOnilNG" OF PROXIMITY VALUES 

One of the basic difficulties facing the user of hierarchical cluster- 
ing is in the need to make a final selection of one clustering result from 
the many that are easily available. Within the behavioral sciences, the 
single-link method usually is ignored as a viable strategy since the solu- 
tions obtained by the method tend to be substantively less interpretable 
than those produced by the complete- link criteria (for instance, coiisult 
Fillenbaum fi Rapoport, 1971, or Johnson, 1967). The biological sciences 
on the other hand, favor the single-link strategy over the complete-link 
and apparently obtain single-link clustering solutions that are usually 
more amenable to substantive evaluation; some controversy does exist, how- 
ever (see Jardine § Sibson, 1971). 

Possibly, one of the reasons for the divergence of empirical opinion 
in the two disciplines regarding the utility of single- link clxastering 
may result from the different types of data usually analyzed. The bio- 
logical sciences concentrate on proximity values that have metric pro- 
perties by defining proximity through appropriate metric measures based 
upon numerical attributes that characterize the objects to be clustered. 
Many of the applications in the behavioral sciences, ho^^/ever, are based 
upon data that fail to satisfy the standard triangle inequality, for ex- 
an5)le, proximity values are commonly produced by direct estimation proce- 
dures from a group of subjects (see Shepard, 1972). If it is true that 
the non-metric properties of the standard behavioral science data will 
generally prevent reasonable hierarchies from being constructed by means 
of the single-link method, alternative single-link related strategies 
that have rather elegant axiomatic justifications (Jardine § Sibson, 1971) 

14 
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may be developed by requiring an initial redefinition o£ proximity to make 
the input data originally more "metric." In fact, the preprocessing of 
proximity measures may be beneficial in obtaining meaningful results from 
other clustering strategies, or for that matter, from alternative data 
analytic techniques such as multidimensional scaling. This initial pre- 
processing will generally require more than the ordinal infoimation in 
the original proximity measures, but at least in the suggestion given be- 
low, any strong interval scale dependencies are avoided; also, if the ori- 
ginal data is already metric, then no change is effected by the preproces- 
sing. 

As one possible suggestion for transforming the original matrix of 
proximity measures, a simple modification of the equation used to prepro- 
cess the proximity data for the r-diameter clustering could be used. 
Specifically, instead of the basic matrix M = P^^^, we could input either 
P^^\..., or p^^""^^ to a clustering strategy, where 

pCk).pCk-l)8pCl)^3nd 
{a..}0{b..}Mc..} = min{a.j^^ 

Essentially, the operation ® forces the triangle inequality to hold in 
stages; the final matrix P^^""^^ has to be metric and contains the shortest 
"distance" between any two objects o^ and Oj in a complete graph with 
edges ^veighted by the proximities in M. These matrix operations are well- 
known in graph theory and are discussed fully in Marshall (1971). 

If the single- link or complete -link clustering procedures are applied 
to P^^^ i M, then the resultant partition depends upon more than the 
ordinal information in the matrix M. Tlie case of P^ ^ is particularly 
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instructive since the dependence on M has been given the specific name 
of "hypemonotone invariant" by Hubert (1972). More precisely, any mono- 
tone increasing transformation of the proximity measures in M is called 
hypermonotone invariant if the transformation also preserves the ordering 
of all first differences between the measures (Suppes 5 Zines, 1963). 
Thus, any clustering procedure that relies only upon the rank order of 
the basic input measures will become a hypeimonotone invariant procedure 
with respect to M when applied to P^^^. The reader is referred to Hubert 
(1972) for a discussion of the value of this type of dependence or to 
Suppes and Zinnes (1963) and Kendall (1962) for a discussion of the basic 
framework that v/ould allow a similar invariance discussion for when 
k > 2. 

As a small and exploratory example of the effect of using the matrices 
P^^^ for k >^ 2 instead of the origijial matrix P^"^^ = M, the prcximity 
values between Rao's 12 Indian castes were reallocated to the 66 object 
pairs at random. This procedure was replicated IC times, and for each 
such reassignment, the single-link and complete-link partition hierarchies 
were obtained using the input matrices P^ \ ^ 5 ^ f^n-1. Using the data 
obtained in this manner. Table 5 gives a goodness-of-fit measure between 
each constructed h ^archy and the input data for selected values of k. 
Specifically, Hubc 1974a) defines the adequacy of a particular partition 
hierarchy through onk order correlation index y; the index is obtained 
between the object j airs ranked on the basis of the input proximity values 
and a reranking of the object pairs produced by the partition hierarchy 
itself. This latter reranking is defined by assigning to each object pair 
the partition level at which the object first appears within the same subsets 



Table 5 here 
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The info nnat ion given in Table 5 leads to several rather interesting 
empirical conjectures: first of all, the hypothesis of randomness in as- 
signing proximity ranks to the object pairs used in generating the sampling 
distributions given by tfubert, appears to be a '"lenient" hypothesis. As 
k increases, the mean y values obtained for ^ increase; consequently, 
as the data become "more metric," and even though they are still based 
upon an initial random assignment of the proximity measures in it be- 

comes much easier to reject the basic randomness hypothesis using the 
sampling distributions generated from P^^-^. Thus a rejection of randomness 
in the assignment of proximity values by a sizable y value appears to be 
a rather weak criterion that a clustering result should easily pass; if 
rejection is not possible, then any further substantive interpretation of 
the partition hierarchy is of dubious worth. Secondly, as the initial 
data become "more metric," the single-link strategy appears to produce a 
more adequate partition hierarchy, at least in terms of Y> and the increase 
in Y is more noticeable for the single-link procedure than for the complete- 
link. In fact, for the "metric" data sets, i.e., P^^^\ the mean y values 
over the 10 replications are fairly close for the two clustering schemes; 
clearly, the single-link proced'^re S is affected more by the non-metric 
nature of the original data. This result suggests that the inability of 
the single-link method to elicit substantively meaningful structures in 
the behavioral sciences may be due in part to the non-metric nature of our 
common proximity measures. Obviously, the chaining characteristics of 
single-link clustering are still present (see Wishart, 1969), but appar- 
ently, there is some mitigation of the problem at least as measured by 
the overall y index. 
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TABLE 1. THEORETICAL DISTRIBUTION OVER PARTITION 
TYPES AT LEVEL 9 FOR n = 12 - OCCUPANCY hWEL 



Partition Type Number of Realizations Probability 

{10,1,1} 66 .001 

{9,2,1} 660 .008 

{8,3,1} 1980 .023 

{8,2,2} 1485 .017 

{7,3,2} 7920 .092 

{7,4,1} 3960 .046 

{6,5,1} 5544 .064 

{6,4,2} 13860 .160 

{6,3,3} 9240 .107 

{5,5,2} 8316 .096 

{5,4,3} 27720 .320 

{4,4,4} 5775 .067 
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TABLE 3. CaiPARISON OF EMPIRICAL AND THEORETICAL DISTRIBUTIONS 
OVER PARTITION TYPES AT LEVEL 9 FOR n = 12 - COLLAPSED OVER PAR- 
TITION TYPES CONTAINING 11 E SAME SIZE LARGEST SUBSET 

Empirical Probabilities 

Theoretical Probability 

Single-link 



Partition Type 



.001 
.008 



Complete-link 
.546 .000 
.209 .003 



.040 



.103 



.029 



.137 



.061 



.101 



.331 



.066 



.328 



.416 



.015 



.458 
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TABLE 4. LBTBL 7 PARTITIONS OBTAINED FROM RAO'S CASTE 
DATA USING r-DIAMETER CLUSTERING - ABBREVIATIONS ARE RAO'S 

r Partition 

1 {{A^,A2,A3,A4},{B^,B2},{Ch,M},{Bh,D},{C^,C2}} 

2 {{A^,A2 , {\ ,^2 '^2^ ' ^^'^^ ' ^^^'^^ ' ^ 

3 {{Bj_,B2.A^,A2,A3,A4},{Ch,M},{Bh,D},[C^},{C2}} 

4 {{BpB2,ApA2,A3,A4},{Ch,M},{Bh,D},{C^},{C2}} 

5 { {B^ , B2 ,A^ , A2 , A3 ,A4 ,Cn,M} , {Bh} , {D} , {C^ } , {C2 } } 



ERIC 



24 



23 



TABLE 5. SELECTED GOCDNESS-OF-FIT y VALUES FOR SI.\ - 
LINK AND CCMPLETE-LINK CLUSTERING USING THE MATRICES P^^- 
SAMPLE SIZE IS 10 



Single-l-'nk Complete-link 

Data Set 



k: 


1 


2 


4 


11 


1 


2 


4 


11 


1 


.13 


.39 


.59 


.59 


.39 


.39 


.70 


.70 








• DO 


. OO 




. OO 






3 


.20 


.49 


.72 


.73 


.36 


.51 


•68 


.69 


4 


.oi 


.53 




fin 
. OU 


• 4Z 


.04 






5 


.25 


.46 


.60 


.60 


.49 


.60 


.66 


.66 


6 


.26 


.53 


.59 


.59 


.38 


.64 


.54 


.54 


7 


.13 


,54 


.73 


.73 


.33 


.44 


.75 


.75 


8 


.13 


.53 


.62 


.62 


.40 


.64 


.68 


.68 


9 


.41 


.72 


.82 


.82 


.40 


.51 


.78 


.78 


10 


.47 


.80 


.85 


.85 


.29 


.77 


.85 


.85 


Means: 


.26 


.55 


.68 


.68 


.39 


.57 


.65 


.65 


Lginal Data: 


.50 


.62 


.64 


.64 


.48 


.65 


.68 


.68 



er|c 



2S 



